Année : 2017
Lieu de publication de l'article :

Résumé de l'article

The widespread use of online discussion forums in educationalsettings provides a rich source of data for researchers interestedin how collaboration and interaction can foster effective learning.Such online behaviour can be understood through the Communityof Inquiry framework, and the cognitive presence construct inparticular can be used to characterise the depth of a student’s criticalengagement with course material. Automated methods have beendeveloped to support this task, but many studies used small datasets, and there have been few replication studies.In this work, we present findings related to the robustness andgeneralisability of automated classification methods for detectingcognitive presence in discussion forum transcripts. We closely ex-amined one published state-of-the-art model, comparing differentapproaches to managing unbalanced classes in the data. By demon-strating how commonly-used data preprocessing practices can leadto over-optimistic results, we contribute to the development of thefield so that the results of automated content analysis can be usedwith confidence.CCS CONCEPTS• Computing methodologies → Cross-validation; Supervisedlearning by classification; • Applied computing → Education;KEYWORDSreplication; data contamination; Community of Inquiry; cognitivepresenceACM Reference Format:Elaine Farrow, Johanna Moore, and Dragan Gašević. 2019. Analysing dis-cussion forum data: a replication study avoiding data contamination . InThe 9th International Learning Analytics & Knowledge Conference (LAK19),March 4–8, 2019, Tempe, AZ, USA. ACM, New York, NY, USA, 10 pages.https://doi.org/10.1145/3303772.3303779

Mots-clés

Experience,Best practice,students,text,features,indicate,research,classifier,general,science,computing,based,level,acm,related,relate,natural,following,new,way,high,use,Code,data,https,http,results,result,messages,message,discussions,models,learning,unbalanced classes,presences,researchers,course,courses,predictive feature,cognitive presence,table,org,education,large class sizes,replications,online,replication study,automating,studies,studying,predict,educational settings,work,works,time,timely,times,classifiers,classify,sample,method,sets,set,final,experiences,approach,practical,forums,sciences,training,train,useful,usefulness,evaluating,evaluated,reported,splitting,split,values,human,humans,validation,social,size,compare,comparable,preprocessing practices,counting,performance,perform,classification,com,number,content analysis,generally,value assigned,test,tested,pre,pres,resolution,cases,case,cross,developed,development,developing,develop,contamination,folded,fold,tools,tool,exploration,levels,phases,phase,agreement,transcripts,transcript,measuring,measures,measure,achieved,achieve,networks,network,analyse,coded,recent,allow,overall,common,issues,issue,match,ing schemes,manual coding,ways,dard,rights,stu,importance,important,copies,copy,ine,adopted,manually,facilitation,means,mean,face,matter,state,held,multi,triggering,scheme indicates,broader,recom,applied,higher,role,optimistic,relatively,relative,requires prior,required,pitfalls,aspects,aspect,directly,chains,chain,coders,referred,label,labelling,indicators,indications,indicating,